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Abstract 

A valued constraint satisfaction problem (VCSP) instance (F, II, w) is a set of variables V 
with a set of constraints II weighted by w. Given a VCSP instance, we are interested in a 
re-weighted sub-instance (F, U' C n,w') such that preserves the value of the given instance 
(under every assignment to the variables) within factor 1 ± e. A well-studied special case is cut 
sparsification in graphs, which has found various applications. We show that a VCSP instance 
consisting of a single boolean predicate P{x,y) (e.g., for cut, P = XOR) can be sparsified into 
0(|F|/e^) constraints if and only if the number of inputs that satisfy P is anything but one (i.e., 
|P“^(1)| ^ 1). Furthermore, this sparsity bound is tight unless P is a relatively trivial predicate. 
We conclude that also systems of 2SAT (or 2LIN) constraints can be sparsified. 


1 Introduction 

The seminal work of Bencziir and Karger [BK96] showed that every edge-weighted undirected 
graph G = {V,E,w) admits cut-sparsihcation within factor (H-e) using 0(e“^n logn) edges, where 
we denote throughout n = |F|. To state it more precisely, assume that edge-weights are always 
non-negative and let CutG(5') denote the total weight of edges in G that have exactly one endpoint 
in S. Then for every such G and e G (0,1), there is a re-weighted subgraph = (F, C E, We) 
with < 0(e“^n log n) edges, such that 

V5CF, Cuta(S) G (l±e)-CutG(5), (1) 

and moreover, such G^ can be computed efficiently. 

This sparsihcation methodology turned out to be very influential. The original motivation was to 
speed up algorithms for cut problems - one can compute a cut sparsifier of the input graph and then 
solve an optimization problem on the sparsiher - and indeed this has been a tremendously effective 
approach, see e.g. [BK96, BK02, KL02, She09, MadlO]. Another application of this remarkable 
notion is to reduce space requirement, either when storing the graph or in streaming algorithms 
[AG09]. In fact, followup work offered several refinements, improvements, and extensions (such 
as to spectral sparsihcation or to cuts in hypergraphs, which in turn have more applications) see 
e.g. [ST04, STll, SSll, dCHSll, FHHPll, KP12, NR13, BSS14, KK15]. The current bound for 
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cut sparsification is 0{nle^) edges, proved by Batson, Spielman and Srivastava [BSS14], and it is 
known to be tight [ACK+15]. 

We study the analogous problem of sparsifying Constraint Satisfaction Problems (abbreviated 
CSPs), which was raised in [KK15, Section 4] and goes as follows. Given a set of constraints on n 
variables, the goal is to construct a sparse sub-instance, that has approximately the same value as 
the original instance under every possible assignment, see Section 2 for a formal dehnition. Such 
sparsihcation of CSPs can be used to reduce storage space and running time of many algorithms. 

We restrict our attention to two-variable constraints (i.e., of arity 2) over boolean domain (i.e. 
alphabet of size 2). To simplify matters even further we shall start with the case where all the 
constraints use the same predicate P : {0,1}^ —)• {0,1}. This restricted case of CSP sparsihcation 
already generalizes cut-sparsihcation — simply represent every vertex v G V hy a variable Xy, and 
every edge {v, u) £ E hy the constraint Xy 7 ^ Xy 

Observe that such CSPs capture also other interesting graph problems, such as the uncut edges 
(using the predicate Xy = Xy), covered edges (using the predicate Xy V Xy) or the directed-cut 
edges (using the predicate Xy A ^Xy). Even though these graph problems are well-known and 
extensively studied, we are not aware of any sparsihcation results for them, and at a hrst glance 
such sparsihcation may even seem surprising, because these problems do not have the combinatorial 
structure exploited by [BK96] (a bound on the number of approximately minimum cuts), or the 
linear-algebraic description used by [SSll, BSS14] (as quadratic forms over Laplacian matrices). 

Results. For CSPs consisting of a single predicate P : {0,1}^ —)• {0,1}, we show in Theorem 3.7 
that a (1 -|- e)-sparsiher of size 0(n/e^) always exists if and only if |P“^(1)| 7 ^ 1 (i.e., P has 0,2,3 or 
4 satisfying inputs). Observe that the latter condition includes the two graphical examples above of 
uncut edges and covered edges, but excludes directed-cut edges. We further show in Theorem 4.1 
that our sparsity bound above is tight, except for some relatively trivial predicates P. We then 
build on our sparsihcation result in Section 5 to obtain (1 -|- e)-sparsihers for other CSPs, including 
2SAT (which uses 4 predicate types) and 2LIN (which uses 2 predicate types). 

Finally, we explore future directions, such as more general predicates and a generalization of 
the sparsihcation paradigm to sketching schemes. In particular, we see that the above dichotomy 
according to number of satisfying inputs to the predicate extends to sketching. 

2 Two-Variable Boolean Predicates and Digraphs 

A predicate is a function P ; {0,1}^ —>■ {0,1} (recall we restrict ourselves throughout to two variables 
and a boolean domain). Given a set of variables V, a constraint {{v,u), P) consists of a predicate 
P and an ordered pair {v,u) of variables from V. For an assignment A :V ^ {0,1}, we say that 
A satisfies the constraint whenever P(A(u), A(u)) = 1. A VCSP (Valued Constraint Satisfaction 
Problem) instance X is a triple (V, FI, rc), where V is a set of variables, 11 is a set of constraints 
over V (each of the form vrj = {{vi,Ui),pi)), and rc : B —>■ M_|_ is a weight function. The value of an 
assignment A : V —)• { 0 , 1 } is the total weight of the satished constraints, i.e., 

Valx(A) := ^ w{-Ki) ■ pi{A{vi), A{ui)). 

TTi£U 

For e G (0,1), an e-sparsifier of X is a (re-weighted) sub-instance X^ = {V, C B, w^) where 

VA ; V ^ {0,1}, Valx,(A) G (1 ± e) • Valx(A). 
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Figure 1: All possible predicates P : {0,1}^ —)■ {0,1}, where blank cells denote value 0. Predicates 
Ox, xO, xl, lx are determined by a single variable. Predicates 01, Dicut, 10, 01 are satisfied by a single 
assignment or all but a single one. 


The goal is to minimize the number of constraints, i.e., |ne|. There are 16 different predicates 
P : {0,1}^ —)> {0,1}, which are listed in Figure 1 with names for easy reference. 

We first focus on the case where all the constraints in IT use the same predicate P,^, in which 
case we can represent the VCSP X by an edge-weighted digraph = (P, E, w). Each variable in V 
is represented by a vertex, and each constraint over the pair (v,u) will be represented by a directed 
edge from v to u, with the same weight as the constraint (formally, E = {(v,u) | {{v,u), P) G FI}, 
and abusing notation set edge weights w{v,u) = w{{{v,u),P))). This transformation preserves all 
the information about the VCSP and allows us to make reductions between VCSPs with different 
predicates P as their sole predicate. 

Given a digraph G, a predicate P and a subset 5 C P, define 

^0(5) := ^ P(ls(u),l5(u)) •u;((u,'u)), 

where denotes the indicator function. For example, applying this definition to the cut predicate 
Cut : (x,y) we have 

CutciS) = ^ Cut(ls(u), l 5 (u)) • u;((u,n)) = ^ |l 5 (u) - l 5 (u)| • w((u, n)), 

{v,u)^E {v,u)^E 

which is just the total weight of the edges crossing the cut S. This matches the definition we gave 
in the introduction, except for the technical subtlety that G is now a directed graph, which makes 
no difference for symmetric predicates like Cut. We shall assume henceforth that G is directed. 

We shall say that a sub-instance Ge is an e-P-sparsifier of G if 

V5CP, PG,(5)G(l±e)-PG(S). 

Observe that given an assignment A for the variables P, we can set Sa ■= {u \ A{u) = 1}. It then 
holds that Vali(A) = Pgi(5a), where G^ is the appropriate digraph for the VCSP. As there a 
bijection between such VCSPs and digraphs, we conclude 

Observation 2.1. The existence of an e-P-sparsiher G^ = {V,Ef:,w^) for G^ implies the existence 
of an e-sparsifier X^ forX with \Ef:\ constraints. 

^The collection of predicates used in a VCSP is sometimes called its signature. In this paper we mainly deal with 
VCSPs whose signature is of size one. 
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Note that the converse is true as well, i.e., an e-sparsifier for X implies the existence of e-P- 
sparsifier for Gx of size |ne|. From now on, we focus on finding an e-P-sparsiher for an arbitrary 
digraph G (for different choices of the predicate P). 

3 A Single Predicate 

In this section we go over all the predicates P : {0,1}^ —>• {0,1} and classify them into sparsifiable and 
non-sparsifiable predicates, see Theorems 3.5, 3.6, and 3.7. For simplicity, we state our sparsification 
results as existential, but in fact all these sparsifiers can be computed in polynomial time. Our 
main technique is a simple graph transformation, which seems to be very well-known but in other 
contexts. We find it surprising that rather different predicates can be analyzed so easily by applying 
the same elementary transformation. 

In our classification, we appeal to two basic predicates, the first of which is Cut, which is already 
known to be sparsifiable. 

Theorem 3.1 ([BSSlll). For every diqraph G and parameter e G (0,1), there is an e-Cut-sparsifier 
forG withO{\V\/e‘^) edges. 

Our second basic predicate is the predicate And, which behaves significantly different. We call a 
digraph G = {V, E) strongly asymmetric if for every [v, u) G FI it holds that (u, v) ^ E. 

Theorem 3.2. For every strongly asymmetric digraph G = (17, E, w) with strictly positive weights 
and e G (0,1), every e-And-sparsifier G^ = {V,Ef:,We) must satisfy Fig = E. 

Proof. Let Gg = (17, Fig, tCg) be such a sparsifier, i.e., for every 5 C 17 it holds that AndG'^(5) G 
(lie) - AndG(«S'). Then for every e = {v,u) G E we must have {v, u) G Fig, as otherwise for the set 
S = {u, u} it will hold that AndG'g({'U, r’}) = 0 while AndG({^i,'v}) = w{e) > 0, a contradiction. □ 

Remark 3.3. For every digraph (which is not necessarily strongly asymmetric), the same proof 
shows that |Flg| > \\E\. 

Remark 3.4. Our definition of an e-P-sparsifier requires Gg to he a subgraph of G, but we can 
state Theorem 3.2 in a more general way: For every digraph Gg = (17, Fig, tUg) (not necessarily a 
subgraph) such that every 5 C 17 satisfies Andc^iS) G (1 ± e) • AndciS) necessarily Fig agrees with 
E up to the direetions of the edges. 

Next, we show that every other predicates is similar either to Cut or to And in terms of 
sparsifability. We describe a reduction that will be useful to show both sparsifability and non- 
sparsifability. (This reduction is based on a well-known transformation of a given graph, called 
the “bipartite double cover”, see e.g. [BHM80], although we are not aware of its use in the same 
way.) Let 7 be a function that maps a digraph G = (V,E,w) where 17 = {vi,V 2 ,... ,Vn} to a 
digraph 7 (G) = {V^,E'^,w'^) where = {v-n,.. . ,u_i,ui,.. .,Vn}, E'^ = {{vi,v-j) \ {vi,Vj) G FI}, 
w'^{{vi,v-j)) = w{{vi,Vj)). For every subset S' C 17, we introduce the notation —S := {v-i \ Vi G S'}, 
S' := {uj I Uj G 17 \ S'} and —S' := {v-i | Uj G 17 \ S}. Figure 2 illustrates the effect of 7 on an 
arbitrary set S. 

Theorem 3.5. For every digraph G = {V,E,w) and e G (0,1) there is a sub-digraph Gg with 
0(|I7|/e^) edges, such that for every predicate Pg {Cut, unCut, Or, nAnd, 10,01, x0,xl, Ox, lx, 1,0}, 
the digraph Gg is an e-P-sparsifier of G. (Note that Gg does not depend on P.) 
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Figure 2: The mapping 7 applied on G and its effect on an arbitrary S GV. For example, an edge from 
Vi & S to Vj S 5 is represented by an arrow of type 3, and becomes in 7 (G) an edge from Vi G S to V-j G —S. 


Proof. Given G and e, first construct 7 (G) as above. Next, apply Theorem 3.1 to obtain for 
7 (G) a cut-sparsifier 7 (G)e = {V'^,E2 C which contains 0(|F^|/e^) = 0{\V\/e^) edges. 

Now construct a digraph G^ = {V,Ef:,We) where E^ = {{vi,Vj) \ {vi,v-j) G Ef} and We{vi,Vj) = 
w7{vi,v-j). Observe that ^{0^) = 7 (G)e, i.e. if we apply 7 on G^ we get exactly 7 (G)e. 

Now suppose that for a predicate P, there is a function /p : 2^ —)■ such that for every 

digraph H on the vertex set V, it holds that 

V5CG, Pj,(5) = Cu^(^)(/p(S)). ( 2 ) 

Then we could apply (2) twice, first to G^ and then to G, and obtain that 

C G, Pa(5) = Cut.,(G),(/p(5)) G (1 ± e) • Cut,(G)(/p(5)) = (1 ± e) • Pg(5). 

Hence, the existence of such a function fp implies that G^ is an e-P-sparsifier. And indeed, we can 
show such fp for some predicates P, as follows. 

• funCut{S) = SGS; 

• fcut{S) = SG-S-, 

. fo4S) = S-, 

• fxo{S) = -S; 

• U{S) = s- 

. /i,(S) = 5; 

• f^(S) = 5 U 5; and 

• foiS) = 0 . 

To verify that funCutiS) = SUS satishes Equation 2 , i.e., that unCutp(S') = Cut.yi^H){S U S), observe 
that both sides consist exactly of the edges of types 1 and 2 in Figure 2. The other predicates can 
be easily verihed similarly, which completes the proof for all P G {Cut, unCut, Ox, xO, xl, lx, 1,0}. 

To show that G^ is a sparsifier also for predicates P G {Or, nAnd, 10, 01} we need a slightly more 
general argument. Suppose that for a predicate P, there are functions /p, /|>, /J- : 2 ^^ —)• such 
that for every digraph H on the vertex set V, 

Ph(S) = i [Cut.,(^)(/i (5)) + Cut.,(p)(/|.(5)) + (5))] . (3) 
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Then we could apply (3) twice, first to and then to G, and obtain that 


PgAS) = i[Cut^(G),(/^(5)) + Cut^(G).(/l(5)) + Cut^(G)J/l(S))^ 

G (1 ± e) • i [Cut^(G)(/i,(S)) + Cut^(G)(/l-(*S)) + Cut^(G)(/|.(5))] 

= (1 ± e) • Pg(5’). 

Hence, the existence of such three functions will imply that is an e-P-sparsifier. And indeed, we 
let 

. flXS) = S, fliS) = -S, = SU-S; 

• /n'And(5) = /n^And(S) = "S, = S U S', 

• 4(5) = 5, 4(5) = -5, 4(5) = 5 U -5; and 
. 4(5) = 5, 4(5) = -5, 4(5) = 5 U -5. 

To verify that /q,., /q^, Jq^ satisfies Equation 3, observe that both sides consist exactly of the edges 
of types 1,3,4 in Figure 2. The other predicates can be easily verified similarly, which completes 
the proof for all P G {Or, nAnd, 10,01}. □ 

Next, we use 7 for a reductions from And to all the remaining predicates. In particular it will 
imply their “resistance to sparsification”. 

Theorem 3.6. Given parameters n and m < ( 2 ), there is a digraph G = {V,E,w) with 2n vertices 
and m edges such that for every e G (0,1) and every predicate P G {nOr, 01, Dicut, And}, for every 
e-P-sparsifier G^ = {V,Ef:,We) of G it holds that that Eg = E. (Note that G does not depend on P.) 

Proof. Let G = {V, E, w) be an arbitrary strongly asymmetric digraph with n vertices, m edges and 
strictly positive weights. Let 7 (G) be the digraph constructed by our reduction. Note that 7 (G) 
consist of 2n vertices and m edges. 7 (G) will be the digraph for which we will prove the theorem. 

Fix some predicate P. Let 7 (G)e = {V^,E2 C E^,w7) be some e-P-sparsifier for 7 (G). Let 
Ge = {V, E^,We) be a digraph where E^ = {(uj, Vj) \ {vi, V-j) G Ef] and ((uj, vj)) = w] {{vi, V-j)). 
Note that 7 (Ge) = 7 (G)e. 

Now suppose that there is a function /p : 2^ —>■ 2^^ such that for every digraph H on the vertex 
set V, it holds that 

V5CE, Andp(5) = P.,(p)(/p(5)). (4) 

Then we could apply (4) twice, first to G^ and then to G, and obtain that 

V5 C E, AndG.(5) = P.,(g).(/p(5)) G (1 ± e) • P^(g)(/p( 5)) = (1 ± e) • AndG(5). 

Hence, assuming such a function / exists, Gg is an e-And-sparsifier for G. According to Theorem 3.2, 
necessarily = E, and in particular Ef = E'^. 

Hence, The existence of such functions fp for all P G {nOr, 01, Dicut, And} will imply our theorem. 
And indeed, we let 


W5) = 5U-5; 
/nOr(5) = 5U-5; 
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• foicutiS) = 5 U -S; and 
. /oi(5) = 5U-5. 

To verify that foicutiS) = S U —S satisfies Equation 4, observe that both sides consist exactly 
of the edges of type 1 in Figure 2. The other predicates can be easily verified similarly. □ 

We conclude our main theorem, which basically puts together Theorems 3.5 and 3.6. 

Theorem 3.7. Let P be a binary predicate, and let e £ (0,1) be some parameter. 

• If P has a single “1” in its truth table then there exist a VCSP X = (E, II, rc) with a single 
predicate P, such that every e-P-sparsifier ofX will have n(|Ep) constraints. 

• If P does not has a single “1” in its truth table then for every VCSP I = {V, II, w) with single 
predicate P, there exists an e-P-sparsifier with O (|E|/e^) constraints. 


4 Lower Bounds (for a Single Predicate) 

In this section we will show that Theorem 3.5 is tight. More precisely, we will show that for 
every P G {Cut, unCut, Or, nAnd, 10,01}, there exists an n-vertex graph G such that every e-P- 
sparsifier Ge of G must contain 0(n/e^) edges. ^ The first step was done by [ACK+15], who showed 
that Theorem 3.1 is tight, i.e., for every n and e G (l/\/n, 1), there exists n-vertex graph G 
such that every e-Cut-sparsifier of G must contain VL{n/e^) edges. Using our reduction 7 in 
similar manner to Theorem 3.5, this lower bound can be extended to unCut based on the fact that 
CutG'(S') = unCut.^(( 5 ) (S U —*?)• However, 7 fails to extend the lower bound to predicates with three 
I’s in their truth table. To this end, we will define sketching schemes, a variation of sparsification 
where the goal is to maintain the approximate value of every assignment using a small data structure, 
possibly without any combinatorial structure, see definition below. We will use a lower bound on 
the sketch size of Cut from [ACK'*'15] to prove lower bound on the number of edges in a sparsifier 
(and also on the sketch size) for OR. The extension to other predicates with three I’s in their truth 
table is straightforward using 7 . Sketching is interesting for its own, and we have further discussion 
and lower bounds regarding sketching in Section 6.3. 

Formally, a sketching scheme (or a sketch in short) is a pair of algorithms (sk, est). Given a 
weighted digraph G = {V,E,w) and a predicate P, algorithm sk returns a string sk^ (intuitively, 
a short encoding of the instance). Given skj and a subset 5 C U, algorithm est returns a value 
(without looking at G) that estimates Pg'(S'). We say that it is an e-P-sketching-scheme if for 
every digraph G, and for every subset S C V, est(skG',S') G (lie) - Pg'(*S'). The sketch-size is 
maxc I skc |, the maximal length of the encoding string over all the digraphs with n variables, often 
measured in bits, sk might be probabilistic algorithm, but for our purposes it is enough to think 
only on the deterministic case. Note that an algorithm for constructing e-sparsifiers always provides 
an e-sketching-scheme, where the sketch-size is asymptotically equal to the number of constraints 
in the constructed sparsifiers when measured in machine words (and up to logarithmic factors 
when measured in bits). Sparsification is advantageous over general sketching as it preserves the 
combinatorial structure of the problem. Nevertheless, one may be interested in constructing sketches 
as they may potentially require signihcantly smaller storage. 

^The other predicates {a;0,xl,0a:, lx, 1,0}, are kind of trivial in the sense of sparsification. 0 sparsified by the 
empty graph. 1 can be sparsified using a single edge. {x0,xl,0x, lx} could be sparsified using n edges. 
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Theorem 4.1. Fix a predicate P G {Cut, unCut, Or, nAnd, 10}, an integer n and e G {l/^/n, 1). The 
sketch-size of every e-P-sketching-scheme on n variables is Pl{n/e^). Moreover, there is an n-vertex 
digraph G, such that every e-P-sparsifier of G has ri(n/e^) edges. 

Proof. We follow the line-of-proof of Theorems 4.1 and 4.2 in [ACK'''15]. Specifically, they show that 
the sketch-size of every e-Cut-sketching-scheme is Q,{n/e^) bits, by proving that a certain family T" of 
n-vertex graphs is hard to sketch, and consequently to sparsify. By similar arguments to Theorem 3.5, 
this lower bound easily extends to unCut. Indeed, recall that CutG'(5') = unCu^(Q) (S' U —S), and 
thus a e-unCut-sparsifier (or sketch) for 7 (G) yields an e-Cut-sparsifier (or sketch) for G with the 
same number of edges (size). 

Once we prove the lower bound for predicate OR, a reduction from OR using 7 will extend it 
also to nAnd, 10 and 01, because 


OrG(S) = nAnd^(G)(S U -S) = 01^(g)(S U -S) = 10^(g)(S U -S). (5) 

We will thus focus on the predicate OR. As it is symmetric predicate, we can work with graphs rather 
then digraphs. The main observation in our proof is that for every undirected graph G = {V, E, w), 
if degG(r') denotes the degree of vertex v, then 

VS C V, CutG(S) = 2 • ORg(S) - degG(^;). (6) 

v£S 

The graph family E consists of graphs G constructed as follows. Let si, ..., 3^/2 C {0,1}^/*^^ be 
balanced 1/e^ bit-strings (i.e., each Si has normalized Hamming weight exactly 1/2), and let the 
graph G be a disjoint union of the graphs {Gj \ j G [e^n/2]}, where each Gj is a bipartite graph, 
whose two sides, each of size 1/e^, are denoted L{Gj) and R{Gj). The edges of G are determined 
by si,..., where each bit string Sj is indicates the adjacency between vertex i G UjL(Gj) and 
the vertices in the respective R{Gj). They further observe (in Theorem 4.2) that the lower bound 
holds even if the sketching scheme is relaxed as follows: 

1 . The estimation is required only for cut queries contained in a single Gj, namely, cut queries 
5 U T where S C L{Gj) and T C R{Gj) for the same j. 

2. The estimation achieves additive error p/e^, where p = 10““^ (instead of multiplicative error 
lie). 

To prove a sketch-size lower bound for a (/ie)-OR-sketching-scheme (sk®^, est®^), we assume it 
has sketch-size s = s{n,e) bits, and use it to construct a Cut-sketching-scheme (sk''“'^, est*“‘^’^) that 
achieves the estimation properties 1 and 2 on graphs of the aforementioned form, and has sketch-size 
s -|- 2nlog(l/e) bits. Then by [ACK+IS], this sketch-size must be Fl{n/€^), and we conclude that 
s = Pl{n/e^) as required. 

Given a graph G n T, let be a concatenation of sk®^ and a list of all vertex degrees in G. 
The degrees in G are bounded by 1/e^, hence the size of is indeed s -|- 2nlog(l/e) bits. Given 
a cut query 5 U T contained in some Gj, dehne the estimation algorithm (which we now construct 
for Cut) to be 


est*““'^(skG‘^’^, S' U T) := 2 • est°^(skG^, S U T) — ^ degG(r')- 


(7) 


Let us analyze the error of this estimate. First, observe that as in each Gj there are precisely 
^ edges, ORg{S LiT) < and thus 

est°'^(skg^, 5 U T) e (1 ± /re) • 0 Rg(5 U T) C ORg{S U T) ± ^ . 

Plugging this estimate into (7) and then recalling our initial observation (6), we obtain as desired 

est^"^(skg"gSur) E2-ORG(5ur)±4- ^^Sg(^) 

^ vesuT 
= CutG(SuT)±^ . 

To prove a lower bound on the size of an OR-sparsifier, we follow the argument in [ACK'’'15, 
Theorem 4.2], which shows that given an e-Cut-sparsifier Gg with s = s(n,e) edges for a graph 
G £ T, there is a Cut-sparsifier G^ of G^, with additive error ^/2e^, such that has only integer 
weights and henceforth can be encoded using + log(e“^n/s))) bits. In fact, there is nothing 

special here about Cut. The same proof will work (with the same properties) for predicate OR, 
assuming a sparsifier is required to be a subgraph (to remove this restriction, just erase all the edges 
between Gj to Gi for i ^ j, which adds only a small additive error). 

Now suppose that every graph G of the form specified above admits a ^e-OR-sparsifier G^ 
with s edges. Then as explained above (about repeating the argument of [ACK+15]) there is a 
graph G^ that sparsifies G^ with additive error /u/2e^, and can be encoded by a string Ig of size 
0(slog(e“^n/s)) bits (recall that /U is a constant). Use it to construct a Cut-sketching-scheme with 
additive error ///e^ as follows. Given the graph G, set skg“'^ to be the concatenation of Ig and a list 
of the degrees of all the vertices in G. Then |Xg| = 0(slog(e“^n/s)) -|- 2nlog(l/e). For a cut query 
S' U T contained in some Gj, define the estimation algorithm (using the OR sparsifier) to be 

est^^Hskg^SSUT) :=2-0RG,(Sur)- degG(u). 

veSuT 

Then we can again analyze it by plugging the above error bounds and then using (6), 

est^^Hskg^^SU^) g2.ORg,(S ur)±^- ^ degG(u) 

^ vesuT 

G2.0RG(sur)±4- E 

^ veSuT 
= CutG(SuT) ± 4 • 

By [ACK'''15], the sketch-size must be |2 g| = 0(n/e^), hence s = 0(n/e^) (for at least one graph 
G G T") as required. □ 

5 Multiple Predicates and Applications 

In this section we extend Theorem 3.5 to VCSPs using multiple types of predicates. In particular, 
we prove sparsifability for some classical problems. Again, our sparsification results are stated as 
existential bounds, but these sparsifiers can actually be computed in polynomial time. 
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Theorem 5.1. For every e G (0,1) and a VCSP (F,H,w) whose constraints {{v,u),P) G II all 
satisfy P ^ {nOr,01, Dicut, And}, there exists an e - sparsifier for Z with 0{\V\/e^) constraints. 

This bound is tight, according to Theorem 4.1. We prove it by a straightforward application 
of Theorem 3.5. Partition Z to disjoint VCSPs according to the predicates in the constraints, and 
then for each sub-VCSP find an e-sparsifier using Theorem 3.5. The union of this sparsifiers is an 
e-sparsifier for Z. A formal proof follows. 

Proof of Theorem 5.1. For each predicate P, let = {vr G II | vr = {{v, u), P)}. Note that {H^} 
forms a partition of II. For each P, let Z^ = {V,H^,w^) where is the restriction of w to 11^. 
Let Z^ = {V,Il^,w^) be an e-P-sparsifier for Z^ with |nf| = 0(|P|/e^) constraints according 
to Theorem 3.5 (recall that P ^ {nOr, 01, Dicut, And}). Set = (14, n^,rc^). He = and 

We = Up We . For every assignment A, 


Valx,(A) = 


WeilTi) ■ Pi{A{vi),A{ui)) 

TTiSlIe 

P TT.enf 

(l±e)-X] w^ {TTi) ■P{A{vi),A{ui)) 

P TT.en-p 

(lie) - w (vTi) • Pi {A{vi), A{ui)) 

TTisn 

(l±e)-Valx(A), 


and note that indeed llle |<0(n/eU. □ 

2SAT (boolean satisfiability problem over constraints with 2 variables) can be viewed as a VCSP 
which uses only the predicates Or, nAnd, 10 and 01. By Theorem 5.1, for every 2SAT formula 
over n variables, and for every e G (0,1), there is a sub-formula with Oinje^^ clauses, such that 
<I> and <1>£ have the same value for every assignment up to factor 1 -|- e? 

2LIN is a system of linear equations (modulo 2), where each equation contains 2 variables and 
has a nonnegative weight. Notice that the equation x -|- y = 1 is a constraint using the Cut predicate 
while the equation x-|-y = 0 is a constraint using the unCut predicate. By Theorem 5.1, if n denotes 
the number of variables, then for every e G (0,1) we can construct a sparsifier with only 0(n/e^) 
equations (i.e., a re-weighted subset of equations, such that on every assignment it agrees with the 
original system up to factor 1 -|- e). 

We note that by our lower bound (Theorem 4.1), there are instances of 2SAT (2LIN) for which 
every e-sparsifier must contain n(n/e^) clauses (equations). 


6 Further Directions 

Based on the past experience of cut sparsification in graphs ~ which has been extremely successful 
in terms of techniques, applications, extensions and mathematical connections - we expect VCSP 

^We use here the version of 2SAT where each clause has weight and every assignment has value rather then the 
version when we only ask weather there an assignment that satisfies all the clauses. 
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sparsification to have many benefits. A challenging direction is to identify which predicates admit 
sparsihcation, and onr results make the first strides in this direction. 

We now discuss potential extensions to our results in the previous sections (which characterize 
two-variable predicates over a boolean alphabet). We first consider predicates with more variables, 
and in particular show sparsification for k-SAT formulas, in Section 6.1. We then consider predicates 
with large alphabets in Section 6 . 2 , showing in particular a sparsiher construction for k-Cut, and 
that linear equations (modulo k > 3) are not sparsifiable. We also consider sketching schemes, 
notable we discnss a more loose sketching model called for-each in Section 6.3. Finally, we study 
spectral sparsihcation for unCut, a notion that preserves some algebraic properties in addition to 
the “uncnts” in Section 6.4. 

6.1 Predicates over more variables and fc-SAT 

It is natural to ask for the best bounds on the size of e-P-sparsihers for different predicates 
P : {0,1}^ —>■ {0,1}. A hrst step towards answering this question was already done by [KK15]. 

Theorem 6.1 ([KK15]). For every hypergraph H = (V,E,w) with hyperedges containing at most r 
vertices, and e G ( 0 , 1 ), there is a re-wighted subhypergraph with 0{n{r + logn)/e^) hyperedges 
such that 

V5 C V, CutH^S) G (1 ± e) • CutniS). 

Here we say that a hyperedge e is cut by S' if 5 H e ^ (i.e., not all the vertices in e 

are in the same side). Observe that Cut is equivalent to the predicate NAE (not all equal). In 
particular Theorem 6.1 implies that for every VCSP using only NAE, there is an e-sparsiher with 
0 (n(r-|-log n)/e^) constraints. 

A fe-SAT is essentially a VCSP that uses only predicates with a single 0 in their truth table. 
[KK15] use Theorem 6.1 to construct an e-sketching-scheme with sketch-size 0{nkje'^) for fc-SAT 
formulas (i.e., only for VCSPs of this particular form). We observe that their sketching scheme can 
be further used to construct an e-sparsfiers, as follows. 

First, recall how the sketching scheme of [KK15] works. Given a fc-SAT formula <!> = {V,C,w) 
(variables, clauses, weight over C), construct a hypergraph H on vertex set V U —V U {/}. We 
associate the literal Vi with vertex Vi, the literal -lUj with vertex v-i, and use / to represent the 
“false”. Each clause becomes a hyperedge consisting of / and (the vertices associated with) the 
literals in C (for example V ^vj V vi 2 becomes {/,us,n_ 7 ,^ 12 }). Observe that given a truth 
assignment A : V —)• {0,1}, if we define Sa '■= {"U | A(u) = 0}, then Val(i)(A) = CutniSA U {/}), 
and using Theorem 6.1 this provides a sketching scheme. Moreover, given an e-Cut-sparsifier 
for H, let $£ be the formula which has only the clauses associated with edges that “survived” the 
sparsihcation, with the same weight. Notice that for every assignment A, 

Va4,(A) = CutH,(5A U {/}) G (1 ± e) • CutniS a U {/}) = (1 ± e) • Vab(A) . 

Theorem 6.2. Given k-SAT formula $ over n variables and parameter e G (0,1), there is an 
e-sparsifier sub-formula (j)f, with 0{n{k + log n)/€'^) clauses. 

In contrast, we are not aware of any nontrivial sparsihcation result for the parity predicate (on 
k >3 boolean variables), and this remains an interesting open problem. 
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6.2 Predicates over larger Alphabets 

Our results deal only with predicates that get two input values in {0,1}. A natural generalization 
is to sparsify a VCSP that uses a predicate over an alphabet of size k, i.e., P : [A:] x [k] —)> {0,1}, 
where [/c] := {0,1,..., A; — 1}. One predicate that we can easily sparsify is NE (not-equal), which is 
satisfied if the two constrained variables have are assigned different values. Indeed, in the graphs 
language, this is called a k-Cut, where the value of a partition (S'o,..., Sk-i) of the vertices is the 
total weight of all edges with endpoints in different parts. It turns out that e-Cut-sparsifier is in 
particular an e-k-Cut-sparsifier, using the following well-known double-counting argument: 

k-CutG. (5o,..., [CutG. (5o,^) + --- + CutG, (5fc_i,^)] 

^ (1 =•= e) • - • [CutG (^o, 5o) -I-1- CutG {Sk-i, S'fc-i)] 

= (I ± e) • k-CutG (S'o,..., S'fc_i) . 

In contrast, linear-equation predicates are non-sparsifiable for alphabet [A:] of size A > 3. 

Specifically, for a G [k], let the predicate Surria be satisfied hy x,y £ [k] iS x + y = a (mod k). 

Then for every positively weighted digraph G = {V,E,w), and every e G (0,1), a G [k], every 
Surria-e-sparsifier Ge = {V,Ee,We) of G must have E = E^. The argument is similar to the proof 
of Theorem 3.2. Assume for contradiction there exist e G E\E^. Choose x,y,z G [k] that satisfy 
x + y = a, however the three sums z + x, z + y, z + z are all not equal to a (modulo A); this is clearly 
possible for A > 4, and easily verified by case analysis for A = 3. Consider an assignment where the 
endpoints of e have values x and y, respectively, and all other vertices have value 2 ;. Under this 
assignment, the value of G is w{e) > 0, while the value of G^ is zero, a contradiction. 

6.3 Sketching 

In Theorem 4.1 we showed that for every predicate P G {Cut, unCut, Or, nAnd, 10}, the sketch-size 
of every e-P-sketching-scheme is U(n/e^). 

Let us now address predicates with a single 1 in their truth table. In the spirit of the proof of 
Theorem 3.2, given encoding skG by an e-And-sketching-scheme we can completely restore the graph 
G. As there are 2 ( 2 ) different graphs, the sketch-size of every e-And-sketching-scheme is at least 
O(n^) bits. Imitating the proof of Theorem 3.6, we can extend this lower bound to Dicut, 01 and 10. 

For-each sketches. In order to reduce storage space of a sketch, one might weaken the require¬ 
ments even further and allow the sketch to give a good approximation only with high probability. A 
for-each sketching scheme is a pair of algorithms (sk, est); algorithm sk is a randomized algorithm 
that given a graph G returns a string skG, whose distribution we denote by 'Dq] algorithm est is 
given such a string skG and a subset S CV, and returns (deterministically) a value est(skG, S). We 
say that it is an [e, 6)-P-sketching-scheme if 

VG= (U,F,u;),V5C U, Pr [est(skG, 5) G (1 ± e) • Pg (5)] > 1 - 5 . 

ska&Va 

[ACK+15] showed that if we consider n-vertex graphs with weights only in the range [1, W], then 
there is an (e, l/poly(n))-Cut-sketching-scheme with sketch-size O (ne“^ • logloglU) bits. Imitating 
Theorem 3.5, we can construct (e, l/poly(n))-P-sketching-scheme with the same sketch-size for every 
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predicate P whose truth table does not have a single 1 (and weights restricted to the range [1, W\). 
A nearly-matching lower bound by [ACK'*“15] shows that for every e G ( 2 /n, 1 / 2 ), every (e, 1/10)- 
Cut-sketching-scheme must have sketch-size Q{n/e). Using 7 , this lower bound can be extended to 
unCut. This technique does not work for predicates with three I’s in their truth table. Fortunately, 
we can duplicate the proof of [ACK“*'15] while replacing Cut by Or and using the fact that for every 
two vertices v,u in the graph G, it holds that Or({u}) -|- Or({u}) — Or({u,tt}) = We 

omit the details of this straightforward argument. A reduction from OR using 7 and equation 5 will 
extend the lower bound also to nAnd,10 and 01. 

Given a sketch skc (i.e., one sample from distribution 'Dq) which encodes an (e, (5)-And-sketching- 
scheme, one can reconstruct every edge of G (every bit of the adjacency matrix) with constant 
probability. Standard information-theoretical arguments (indexing problem) imply that the sketch- 
size of every (e, (5)-And-sketching-scheme is O(n^) bits. Using 7 we can extend this lower bonnd to 
Dicut, 01 and 10. 

6.4 unCut Spectral Sparsifiers 

Given an nndirected n-vertex graph G = (U, E, w), the Laplacian matrix is defined as Lq = Dq — Ag 
where Aq is the adjacency matrix (i.e. Ajj = Wij = w{{vi,Vj})) and Dq is a diagonal matrix 
of degrees (i.e. Di^i = for i 7 ^ j, Dij = 0). For every x G it holds that 

x^Lgx = Yl{vi vj}&E particular, for I 5 the indicator vector of some subset S' C U 

it holds that = Cute(5). A subgraph FT of G is called an e-spectral-sparsifier of G if 

Vx G M"', x^LffX G (1 ± e) • x^Lqx . 

Note that an e-spectral-sparsifier is in particular an e-Cut-sparsifier. Nonetheless, spectral sparsifiers 
preserve additional properties such as the eigenvalues of the Laplacian matrix (approximately). 
[BSS14] showed that every graph admits an e-spectral-sparsifier with 0(n/e^) edges. 

Definition 6.3. Given a graph G, we call Uq = {Dq + Aq) the Negated Laplacian of G. Given a 
subset S CV, let efs GRA be a vector such that (ps,i = 1 if Vi G S and 4)s,i = — 1 otherwise. 

One can verify that for arbitrary x G 

x^Ugx = ^ Wij ■ {xi + Xj)^ 
i<j 

In particular, for every subset S' C U, it holds that 

4>%UG(i)s = 4 • unCutG(S') . 

Next, we will show how we can use Uq to construct an unCut-sparsifier Ge (in alternative way to 
Theorem 3.5) such that has (approximately) the same eigenvalues as Ug- A matrix M G 
is called BSDD [Balanced Symmetric Diagonally Dominant) if M = M* and for every index i, 
\ Adi,j\- Note that Lg and Ug are both BSDD. A matrix M' is governed by M if 
whenever MU ^ Q, also Mjj / 0 and has the same sign. Note that if iF is a subgraph of G then Uh 
is governed by Ug- A matrix M' is called an e-spectral-sparsifier of M if M' is governed by M and 

Vx G M”, x*M'x G (1 ± e) • x*Mx . 

The following was implicitly shown in [AGK'*“15]. 
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Theorem 6.4 ([ACK+15]). Given BSDD matrix M G and parameter e G (0,1), there is an 

e-spectral-sparsifier M' for M where M' is BSDD matrix with 0{n/e‘^) non-zero entries. 

Fix a graph G and parameter e, according to Theorem 6.4, there is a BSDD balanced matrix H 
with 0{nle^) non-zero entries, that governed by Uq which is a e-spectral-sparsifier for Uq- All this 
properties define a unique graph such that = H. In particular is e-unCut-sparsifier of G 
with Oinje^') edges. 
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